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Preface 

The  Research  reported  in  this  document  has  been  made  possible 
through  the  support  and  sponsorship  of  the  U.S.  Government  through 
its  European  Research  Office.  This  report  is  intended  only  for 
the  internal  management  use  of  the  Grantee  and  the  U.S.  Govern¬ 
ment  . 


The  work  was  performed  in  the  period  of  August  1980  to  April 


The  final  technical  report  /I/  describes  three  methods  for  the 
cooperative  extraction  of  line  shaped  objects  from  aerial  photo¬ 
graphs.  The  first  method  calculates  starting  points  for  the 
extraction.  The  two  other  methods  are  the  local  method  and  the 
regional  method  for  line  following.  The  three  algorithms  origi¬ 
nally  were  implemented  in  Standard-FORTRAN  on  a  minicomputer  DEC 
PDP  1 1/70.  Due  to  the  limited  working  storage  the  three  methods 
were  implemented  as  three  separate  program  systems. 

All  programs  have  been  transfered  to  a  VAX  11/780  computer  and 
integrated  into  one  system.  The  extraction  runs  completely  auto¬ 
matically  and  large  image  data  can  be  processed. 

The  following  chapters  describe  the  hardware  configuration,  the 
implemented  software  system  and  the  process  of  the  extraction 
in  large  image  matrices.  A  complete  printout  of  the  computer  pro¬ 
gram  is  added  as  a  separate  document.  A  magnetic  tape  with  all 
source  files  is  available  from  USAETL.  (The  programs  were  imple¬ 
mented  for  the  purposes  of  a  research  project  to  test  the  methods 
for  the  extraction  of  linear  features,  they  are  neither  optimized 
nor  purged) . 

2.  The  FIM  image  processing  system  configuration,  overview  and 
block  diagram  of  the  implemented  software  system  for  the 
automatic  extraction  of  linear  features 


Figure  1  shows  the  FIM  image  processing  system  configuration. 
Originally  the  automatic  extraction  of  linear  features  was  imple¬ 
mented  on  the  PDP  11/70.  During  the  extraction  the  results  are 
displayed  on  a  COMTAL  color  display  with  interactive  control. 

So  the  interpreter  can  stop  and  correct  the  calculations  if 
necessary.  The  image  data  is  digitized  with  the  DICOMED  scanner 
where  images  up  to  6  cm  x  6  cm  can  be  scanned  with  512  x  512, 

1024  x  1024  or  2048  x  2048  pixels. 


Due  to  the  limitations  of  the  PDP  11/70  only  small  parts  of  the 
image  matrices  can  be  stored  in  the  working  storage  which  causes 
a  lot  of  data  transfer.  For  the  same  reason  the  starting  point 
analysis  and  the  local  and  the  regional  method  for  line  following 
had  been  implemented  as  three  separate  program  systems.  Each 
system  consists  of  a  large  amount  of  subroutines  using  overlay 
programming  structures. 

The  cooperation  of  the  three  systems  is  simulated  interactively. 
The  first  stage  of  the  cooperation  is  the  calculation  of  starting 
points.  The  search  for  starting  points  is  limited  to  a  grid  of 
test  lines  which  is  spread  over  the  whole  image.  Then  the  local 
method  extracts  all  parts  of  the  network  of  lines  which  are 
within  reach  of  these  starting  points.  The  extraction  is  contin¬ 
ued  by  the  regional  method  at  the  dead  ends.  The  regional  method 
is  followed  by  the  local  method  etc..  If  none  of  the  methods 
yields  further  results  the  evaluation  of  the  image  is  terminated. 

The  fact  that  during  the  extraction  all  intermediate  results  are 
displayed  on  a  graphic  overlay  of  a  COMTAL  color  display  is  used 
to  prevent  the  multiple  extraction  of  lines.  For  every  new  line 
segment  the  data  of  the  graphic  overlay  corresponding  to  the 
location  of  this  line  segment  are  checked  whether  it  has  already 
been  detected  or  not. 

To  enable  the  cooperation  between  the  three  separate  program 
systems  the  results  are  stored  in  two  data  files  on  disk  storage. 
The  starting  point  method  writes  the  results  on  a  scratch  file 
from  where  the  local  method  can  read  the  starting  points. 

The  local  method  and  the  regional  method  use  the  same  data  file 
appending  new  results  to  the  results  obtained  earlier. 

On  the  VAX  11/780  the  three  separate  systems  are  integrated  into 
one  system  without  overlay  programming  structure.  So  the  coopera¬ 
tion  of  the  three  parts  is  performed  fully  automatically.  At 
first  the  starting  point  method  evaluates  the  grid  of  testlines 
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until  the  first  starting  point  is  detected.  The  control  is 
switched  automatically  to  the  local  method  which  extracts  all 
possible  parts  of  the  network  of  lines.  The  regional  method  tries 
to  continue  the  extraction  at  the  dead  ends  until  object  proper¬ 
ties  acceptable  for  the  local  method  are  detected  again.  After 
evaluating  all  dead  ends  the  control  is  switched  back  to  the 
local  method  etc.  If  the  regional  method  fails  at  all  dead  ends 
the  next  starting  point  is  searched.  For  that  the  starting  point 
method  continues  to  evaluate  the  grid  of  testlines.  The  algorithm 
stops  if  no  more  starting  points  can  be  found  on  the  grid. 

By  calculating  only  one  starting  point  at  a  time,  it  is  assured 
that  no  redundant  starting  points  are  detected  in  one  connected 
part  of  the  network  of  lines.  The  extraction  process  itself  is 
performed  automatically.  The  final  results  are  stored  in  two  data 
files,  one  with  linked  and  one  with  cellular  data  organisation 
/2/.  The  linked  data  organisation  consists  of  polygons  and 
special  header  information;  each  polygon  corresponds  to  a  line 
object.  The  cellular  data  organisation  consists  of  a  byte  matrix 
of  the  same  size  as  the  image  matrix  where  the  lines  (and  special 
attributes)  are  mapped  one  to  one.  During  the  extraction  this 
matrix  is  checked  to  prevent  the  multiple  extraction  of  a  line. 
The  final  results  can  be  printed  on  a  lineprinter,  plotted  on 
the  CALCOMP  plotter  or  displayed  on  a  COMTAL  color  display. 

In  contrast  to  the  PDP  11/70  the  VAX  11/780  allows  processing 
of  large  image  data.  Aerial  photographs  up  to  23  cm  x  23  cm  can 
be  scanned  with  the  OPTRONICS  image  digitizer  with  a  resolution 
of  25  Mm,  50  Mm  or  100  Mm.  The  resulting  image  matrices  are  split 
into  subimages  of  512  x  512  pixels  each  which  are  adequate  to  the 
working  set  and  can  be  stored  at  once  in  the  working  storage  of 
the  VAX.  The  extraction  in  a  subimage  is  performed  as  described 
above.  The  locations  where  lines  are  bounded  by  the  picture 
limits  are  stored  as  socalled  Dead  Ends  at  Picture  Limits 
(DEPLs).  These  DEPLs  are  used  to  continue  the  extraction  in 
neighbouring  subimages.  The  following  block  diagram  (figure  2) 
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demonstrates  the  extraction  process  in  large  image  data  which 
are  split  into  subimages. 


Figure  2:  Block  diagram  for  the  extraction  process  in  large 
image  data. 
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3.  Retails  of  the  software  system 
3. 1  Input  data  and  parameters 

The  input  image  data  is  a  gray  level  matrix  (scanner  data  or 
mostly  digitized  analogous  aerial  imagery).  The  width  of  the  ob¬ 
jects  depends  on  the  resolution  of  the  scanning.  As  described  in 
/ 1 /  the  special  line  detectors  rely  on  a  sophisticated  analysis  of 
gray  level  diagrams  where  they  recognize  and  isolate  the  gray 
level  profile  of  the  objects  cross  section.  The  length  of  the 
profile,  i.e.  the  width  of  the  object,  should  exceed  one  pixel, 
because  peaks  of  one  pixel  can  simply  be  produced  by  noise.  For 
practical  reasons  the  upper  limit  for  the  object  width  is  approx¬ 
imately  ten  pixels  in  the  implementation. 

Most  line  shaped  objects  in  aerial  imagery  are  either  light  or 
dark  compared  with  their  surrounding  in  the  whole  image.  In 
panchromatic  photographs  roads  mostly  stand  out  light  and  rivers 
mostly  are  darker  than  their  surrounding.  In  scanner  data  this 
is  different,  because  the  same  object  can  be  light  in  one 
spectral  band  and  dark  in  another. 

At  the  beginning  of  the  extraction  the  interpreter  enters  the 
name  of  the  image  data  file.  The  resolution  of  the  scanning  is 
coded  in  the  image  file  header.  All  parameters  automatically  get 
standard  default  values  which  are  suitable  for  the  usual  case, 
the  extraction  of  light  main  roads.  If  this  class  of  objects  is 
to  be  extracted  no  further  input  is  necessary.  If  other  classes 
of  objects  are  to  be  extracted,  the  interpreter  has  to  specify 
the  width  of  these  objects  and  whether  they  are  light  or  dark. 

All  other  parameter  values  are  derived  from  these  two  values  and 
the  scanning  resolution.  Additionally  special  parameters  for  the 
extraction  itself  are  obtained  by  analysing  the  surrounding  of 
the  starting  points,  once  they  are  detected.  Thus  the  object 
specific  extraction  of  each  particular  class  of  objects  is 
possible,  e.g.  small  roads  can  be  extracted  while  main  roads, 
highways,  rivers  etc.  are  disregarded. 
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The  goal  of  this  project  was  to  extract  linear  features  from 
parts  of  aerial  images  with  a  sidelength  up  to  20  cm  and  a  scale 
of  1  :  71*  000.  To  obtain  at  least  two  pixels  even  on  small  roads 
the  images  were  scanned  with  the  OPTRONICS  scanner  using  a  dis¬ 
tance  of  25  Mm  between  neighbouring  scanning  points.  This  resolu¬ 
tion  yields  data  files  of  8192  x  8192  pixels  for  every  20,5  cm  x 
20,5  cm  image.  To  profit  best  of  the  computer  memory  working  set 
the  image  matrix  of  8192  x  8192  pixels  was  split  into  17  x  17 
overlapping  submatrices  of  512  x  512  pixels  each.  The  overlapping 
range  between  neighbouring  subimages  is  3 2  pixels.  This  is 
necessary  because  the  DEPLs  consisting  of  a  line  segment  have 
to  be  part  of  both  subimages.  Moreover  in  case  of  a  smaller 
overlapping  it  might  be  necessary  to  change  images  very  often 
to  extract  an  object  close  to  the  picture  limits. 


3- 3  Structure  of  the  output  data 

The  intermediate  result  of  the  extraction  in  a  512  x  512  subimage 
is  a  string  of  coordinates,  where  each  pair  of  coordinates 
corresponds  to  a  point  on  a  line  object.  Mostly  two  successive 
points  are  not  neighbouring  points,  their  distance  depends  on 
the  width  of  a  step  in  the  local  method  or  the  distance  of  the 
intersecting  lines  in  an  Aol  of  the  regional  method. 

The  final  results  are  of  two  different  kinds,  known  as  cellular 
and  linked  data  organisation.  For  the  first  final  result  the  gaps 
between  two  successive  points  are  interpolated.  This  yields 
connected  chains  of  points  along  all  extracted  line  objects. 

These  points  are  stored  in  a  512  x  512  byte-matrix  where  each 
position  corresponds  to  the  same  position  in  the  relative 
512  x  512  image  submatrix.  The  eight  bits  per  matrix  element  can 
be  used  to  code 
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if  this  position  belongs  to  a  line  object, 

if  the  relative  point  in  the  image  is  a  crossing 
between  line  objects,  a  dead  end  or  a  DEPL, 

the  class  of  the  object  (e.g.  road,  highway), 

the  width  of  the  object, 

the  kind  of  contrast  (light  object  or  dark  object), 

if  the  point  was  a  starting  point 

and  which  method  has  performed  the  extraction. 


This  kind  of  data  organisation  suits  well  for  printing  the  final 
results  as  contour  map  on  a  line  printer  or  displaying  them  on 
a  COMTAL  color  display. 

In  the  linked  data  organisation  the  final  results  are  stored  in  a 
reduced  form.  To  do  this  a  polygon  approximation  is  performed 
for  every  line  object,  so  that  only  the  ends  (dead  ends,  DEPLs 
or  crossings)  and  the  corners  of  the  lines  remain.  The  polygons 
are  stored  in  list  form  as  a  string  of  coordinates  where 
different  polygons  are  separated  by  special  coding.  Additional 
information  is  stored  in  headers,  or  pointers  in  the  headers 
indicate  where  the  information  can  be  found.  This  type  of  final 
result  suits  well  for  plotting  the  results  on  the  CALCOMP 
plotter. 


3 • ^  Size  of  the  software  system  and  program  listings 

The  system  for  the  extraction  of  linear  features  is  implemented 
in  Standard  FORTRAN  IV. 


The  three  main  blocks  of  the  system  are 


a)  the  starting  point  analysis, 

14  programs  (FORTRAN  routines), 
approximately  2000  statements, 

b)  the  local  method, 

36  programs  (FORTRAN  routines), 
approximately  5000  statements, 

c)  the  regional  method, 

37  programs  (FORTRAN  routines), 
approximately  2500  statements. 

The  whole  system  consists  of  approximately  9000  statements  in 
89  programs  (FORTRAN  routines). 

A  complete  printout  of  the  computer  program  is  added  as  a  sepa¬ 
rate  document.  A  magnetic  tape  with  all  source  files  is  available 
from  USAETL.  (The  programs  were  implemented  for  the  purposes  of 
a  research  project  to  test  the  methods  for  the  extraction  of 
linear  features,  they  are  neither  optimized  nor  purged). 

The  first  program  is  SUCOUT  which  manages  the  cooperation  of  the 
three  subsystems.  Behind  SUCOUT  and  one  subroutine  called  from 
SUCOUT  the  three  subsystems  follow  separately  where  each  sub¬ 
system  is  ordered  alphabetically. 


The  first  step  to  speed  up  the  complete  process  is  to  provide 
the  image  data  in  digital  form,  e.g.  by  a  multispectral  scanner. 
Thus  the  time  consuming  task  to  digitize  the  photographs  is 
avoided.  The  next  step  is  to  use  special  hardware  and  to  optimize 
the  software. 

The  processing  times  cited  in  chapter  5  of  /I/  were  obtained  with 
FORTRAN  programs  which  were  implemented  to  test  the  methods  for 
the  extraction  of  linear  features  on  a  DEC  PDP  11/70  and  a  DEC 
VAX  11/780.  An  optimisation  of  these  programs  (e.g.  optimisation 
of  the  data  transfer  disk-memory  and  v.v.)  would  yield  time  saving 
of  approximately  twenty  per  cent. 

Most  of  the  computations  can  be  performed  on  an  array  processor 
(e.g.  Floating  Point  Systems  AP  120  B),  where  the  software  has 
to  be  adapted  according  to  the  demands  of  the  AP.  Though  the 
computations  are  highly  data  dependent  and  the  advantages  of  the 
AP  cannot  be  profited  best,  the  run  time  without  I/O  would 
decrease  by  factor  two  to  three.  In  comparison  to  the  VAX  the 
data  transfer  disk-memory  and  v.v.  can  be  speeded  up  by  factor 
two,  if  a  disk  storage  is  connected  directly  to  the  AP.  The  next 
possibility  is  to  use  several  array  processors,  where  each  AP 
processes  one  subimage  of  the  mosaic.  For  example,  the  estimated 
average  processing  time  achievable  with  two  APs  would  be  less 
than  40  minutes  for  evaluating  a  complete  aerial  photograph. 
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